HPC@UCLA: Introduction to the Hoffman2 Cluster

Raffaella D'Auria, PhD

Learning Outcomes

1) what is the Hoffman2 Cluster

2) what can the Hoffman2 Cluster do for you

3) navigating the documentation

4) getting an account on the Hoffman2 Cluster

5) connecting to the Hoffman2 Cluster via terminal and SSH

6) opening graphical applications on the Hoffman2 Cluster

7) unix command line 101

8) Getting work done on the Hoffman2 Cluster

What is the Hoffman2 Cluster

Screen%20Shot%202022-02-07%20at%2012.45.17%20PM.png

Nomenclature

Compute node:

Computing core:

Login node:

Nomenclature (cont'd)

Computing Cluster:

Learning Outcomes 2

1) what is the Hoffman2 Cluster

2) **what can the Hoffman2 Cluster do for you**

3) navigating the documentation

4) getting an account on the Hoffman2 Cluster

5) connecting to the Hoffman2 Cluster via terminal and SSH

6) opening graphical applications on the Hoffman2 Cluster

7) unix command line 101

8) Getting work done on the Hoffman2 Cluster

What can the Hoffman2 Cluster do for you

High Performance Computing

Capacity Computing

What can the Hoffman2 Cluster do for you (cont'd)

Licensed software availability

Licensed software management

User Support/Project Collaboration/Training

Learning Outcomes 3

1) what is the Hoffman2 Cluster

2) what can the Hoffman2 Cluster do for you

3) **navigating the documentation**

4) getting an account on the Hoffman2 Cluster

5) connecting to the Hoffman2 Cluster via terminal and SSH

6) opening graphical applications on the Hoffman2 Cluster

7) unix command line 101

8) Getting work done on the Hoffman2 Cluster

Navigating the documentation

https://www.hoffman2.idre.ucla.edu/

Screen%20Shot%202022-02-07%20at%204.35.52%20PM.png

Learning Outcomes 4

1) what is the Hoffman2 Cluster

2) what can the Hoffman2 Cluster do for you

3) navigating the documentation

4) **getting an account on the Hoffman2 Cluster**

5) connecting to the Hoffman2 Cluster via terminal and SSH

6) opening graphical applications on the Hoffman2 Cluster

7) unix command line 101

8) Getting work done on the Hoffman2 Cluster

Getting an account on the Hoffman2 Cluster

More information are available at: https://www.hoffman2.idre.ucla.edu/Accounts/Accounts.html#

In what follow a demonstration of a registration as a H2C Faculty Sponsor and a H2C user will be show.

Connecting to SIM for the first time

Screenshot%202024-04-02%20at%208.38.05%E2%80%AFPM.png

Registering with SIM as a Hoffman2 Faculty Sponsor

UCLA faculty or researchers that qualifies under UCLA Policy 900 as principal investigators can apply to become Hoffman2 Cluster Faculty Sponsors

Below is a screenshot with the information provided to the SIM registration page by the hypotetical PI, Éuro Bruin, who is applying to become a Hoffman2 Cluster Faculty Sponsor:

Screenshot%202024-04-03%20at%207.48.10%E2%80%AFAM.png

Registering with SIM as a Hoffman2 Faculty Sponsor (cont'd)

Sponsors registration requires the creation of a unix group, all perspective users applying under a PI sponsorship will be part of this unix group.

Screenshot%202024-04-03%20at%207.58.26%E2%80%AFAM.png

Registering with SIM as a Hoffman2 Faculty Sponsor - Approval Step

All Sponsors applications have to be verified and approved, this process generally take about a working day. After selecting the unix group name SIM will display:

Screenshot%202024-04-03%20at%208.02.36%E2%80%AFAM.png

Requesting a user account (for Sponsors or Sponsorees)

SIM currently does not send email notification, Sponsors need to log back into SIM to check the status of their application

Sponsors can sponsors themselves for a user account

After approval Sponsors can create an account for themselves and approve/deny new users requesting their sponsorship:

Screenshot%202024-04-03%20at%208.07.23%E2%80%AFAM.png

Requesting a user account (for Sponsors or Sponsorees) -cont'd

All perspective users, after registration with SIM can request a user account, the first step is to select a Hoffman2 Facutly Sponsor:

Screenshot%202024-04-03%20at%208.25.04%E2%80%AFAM.png

Completing the user cluster account request

Screenshot%202024-04-03%20at%208.34.02%E2%80%AFAM.png

Sponsors account approval or denial

After requesting an account perspective users should remind their selected sponsors to log into SIM and approve or deny their application.

SIM does not send email messages, if the sponsor does not log into SIM accounts requests will not be approved

This is what a PI who has pending account requests would see upon logging onto SIM:

Screenshot%202024-04-03%20at%208.36.42%E2%80%AFAM.png

Setting the password for your account

Upon being approved, users will need to log back into SIM (you will not be notified by SIM that your account has been approved, it is your responsibility to periodically check).

An approved user will see the following menu requesting to set a password for their newly created accounts:

Screenshot%202024-04-03%20at%208.42.00%E2%80%AFAM.png

Password resetting

Use the link: https://sim.idre.ucla.edu/sim/account/view and click on the link: Change the password for <YOURHOFFMAN2USERNAME> on the H2 cluster shown below for user dbruin.

Screenshot%202024-04-03%20at%208.48.18%E2%80%AFAM.png

Managing your account

https://www.hoffman2.idre.ucla.edu/Accounts/Users-managing-your-account.html#password-processes

Screenshot%202023-10-10%20at%209.10.20%E2%80%AFPM.png

https://www.hoffman2.idre.ucla.edu/Accounts/Sponsors-managing-your-group.html#sponsor-responsibilities

Screenshot%202023-10-10%20at%208.57.01%E2%80%AFPM.png

For sponsor: managing users in your group

https://www.hoffman2.idre.ucla.edu/Accounts/Sponsors-managing-your-group.html#managing-your-users

Screenshot%202023-10-10%20at%209.14.37%E2%80%AFPM.png

Learning Outcomes 5

1) what is the Hoffman2 Cluster

2) what can the Hoffman2 Cluster do for you

3) navigating the documentation

4) getting an account on the Hoffman2 Cluster

5) **connecting to the Hoffman2 Cluster via terminal and SSH**

6) opening graphical applications on the Hoffman2 Cluster

7) unix command line 101

8) Getting work done on the Hoffman2 Cluster

Connecting to the Hoffman2 Cluster via terminal and SSH

https://www.hoffman2.idre.ucla.edu/Using-H2/Connecting/Connecting.html

Screen%20Shot%202022-02-07%20at%204.55.02%20PM.png

Connecting via a terminal and an SSH client (from Linux)

https://www.hoffman2.idre.ucla.edu/Using-H2/Connecting/Connecting.html#connecting-via-terminal-and-ssh

Screenshot%202024-04-02%20at%202.55.41%E2%80%AFPM.png

Connecting via a terminal and an SSH client (from Mac)

https://www.hoffman2.idre.ucla.edu/Using-H2/Connecting/Connecting.html#connecting-via-terminal-and-ssh

Screenshot%202024-04-02%20at%202.57.46%E2%80%AFPM.png

Connecting via a terminal and an SSH client (from Windows)

https://www.hoffman2.idre.ucla.edu/Using-H2/Connecting/Connecting.html#connecting-via-terminal-and-ssh

Screenshot%202024-04-02%20at%203.00.36%E2%80%AFPM.png

Connecting via terminal and SSH - Linux, Mac & Windows

on a Windows PowerShell make sure that the version is 5.1 or later with the command:

 $PSVersionTable.PSVersion

on any terminal applications you may run (aside from Putty and Cygwin excluded in this presentation), type

 ssh <YOURHOFFMAN2CLUSTERUSERNAME>@hoffman2.idre.ucla.edu

Connecting via a terminal and SSH for the first time

https://www.hoffman2.idre.ucla.edu/About/FAQ/FAQ.html#connecting-for-the-first-time

WSL2_FIRST_TIME_LOGGING_IN.png

Connecting for the first time hostkey fingerprints

https://www.hoffman2.idre.ucla.edu/About/FAQ/FAQ.html#table-h2-public-host-classes

WSL2-FIRST-TIME-SSH-HOST-FINGERPRINT-CHECK2.png

Connecting via terminal and SSH (cont'd)

WSL_Connected.png

Set up SSH keys to connect to the Hoffman2 Cluster

SSH keys are authentication credentials that can be set up for passwordless authentication (use of a passphrase is strongly recommended).

On a terminal on your local computer generate SSH-key:

 ssh-keygen -f $HOME/.ssh/hoffman2.idre.ucla.edu

On a terminal on your local computer create a $HOME/.ssh/config using, for example, the nano editor:

 nano $HOME/.ssh/config
 bash -c 'nano $HOME/.ssh/config'

On a terminal on your local computer add these lines to your $HOME/.ssh/config:

Host hoffman2
 hostname hoffman2.idre.ucla.edu
 ServerAliveInterval 30
 ServerAliveCountMax 5
 IPQoS throughput
 IdentityFile=~/.ssh/hoffman2.idre.ucla.edu

Set up SSH keys to connect to the Hoffman2 Cluster (cont'd)

copy the key to the Hoffman2 Cluster

 $ ssh-copy-id -i ~/.ssh/hoffman2.idre.ucla.edu <YOURH2CUSERNAME>@hoffman2.idre.ucla.edu

connect to the Hoffman2 Cluster

 $ ssh <YOURHOFFMAN2CLUSTERUSERNAME>@hoffman2

Learning Outcomes 6

1) what is the Hoffman2 Cluster

2) what can the Hoffman2 Cluster do for you

3) navigating the documentation

4) getting an account on the Hoffman2 Cluster

5) connecting to the Hoffman2 Cluster via terminal and SSH

6) **opening graphical applications on the Hoffman2 Cluster**

7) unix command line 101

8) Getting work done on the Hoffman2 Cluster

Opening graphical applications on the Hoffman2 Cluster

X Window Sytems/X11: The software that allow displaying of graphical applications on a unix-like system and that is designed to work on a server-client mode (you can connect to it and forward graphical applications to your local computer)

Example: forward the MATLAB desktop GUI from Hoffman2 to your local computer

Screen%20Shot%202022-02-07%20at%2011.23.57%20PM.png

Opening graphical applications on Linux (Hands-on)

Opening graphical applications on MacOS (Hands-on)

first time only:

defaults write org.macosforge.xquartz.X11 enable_iglx -bool true

connect with:

Opening graphical applications on Windows

no need to install other software then any of the following:

need to install an X11 window server if using either of:

a possible X11 Window server:

Opening graphical applications on Windows using MobaXterm (Hands-on)

Screenshot%202022-02-08%20074952.png

Opening graphical applications on Windows using MobaXterm (cont'd)

Screenshot%202022-02-08%20075046.png

Opening graphical applications on Windows using MobaXterm (cont'd)

Screenshot%202022-02-08%20075130.png

Opening graphical applications on Windows using MobaXterm (cont'd)

login.png

Opening graphical applications on Windows using MobaXterm

xclock.png

Learning Outcomes 7

1) what is the Hoffman2 Cluster

2) what can the Hoffman2 Cluster do for you

3) navigating the documentation

4) getting an account on the Hoffman2 Cluster

5) connecting to the Hoffman2 Cluster via terminal and SSH

6) opening graphical applications on the Hoffman2 Cluster

7) **unix command line 101**

8) Getting work done on the Hoffman2 Cluster

Unix command line 101

https://www.hoffman2.idre.ucla.edu/Using-H2/Command-line/Unix-command-line.html

Screen%20Shot%202022-02-08%20at%208.49.56%20PM.png

Unix command line 101: Navigation (Hands-on)

https://www.hoffman2.idre.ucla.edu/Using-H2/Command-line/Unix-command-line.html#navigation

ls
ls -a
ls -l
ls -lat
ls -latr
pwd
cd $SCRATCH
pwd
timestamp=`date "+%F_%H-%M"`; mkdir $HOME/H2HH_$timestamp; cd $HOME/H2HH_$timestamp
pwd
cd
rmdir $HOME/H2HH_$timestamp
ls $HOME/H2HH_$timestamp

Move to Hands-on notebook

Unix command line 101: Environmental Variables (Hands-on)

https://www.hoffman2.idre.ucla.edu/Using-H2/Command-line/Unix-command-line.html#unix-environment-environmental-variables

$PATH
$LD_LIBRARY_PATH
$HOME
$SCRATCH  # Hoffman2-specific
$SHELL
echo $HOME

Move to Hands-on notebook

Unix command line 101: Working with files (Hands-on)

https://www.hoffman2.idre.ucla.edu/Using-H2/Command-line/Unix-command-line.html#working-with-files

Move to Hands-on notebook

Unix command line 101: Editing files (Hands-on)

https://www.hoffman2.idre.ucla.edu/Using-H2/Command-line/Unix-command-line.html#editing-files

Learning Outcomes 8

1) what is the Hoffman2 Cluster

2) what can the Hoffman2 Cluster do for you

3) navigating the documentation

4) getting an account on the Hoffman2 Cluster

5) connecting to the Hoffman2 Cluster via terminal and SSH

6) opening graphical applications on the Hoffman2 Cluster

7) unix command line 101

8) **Getting work done on the Hoffman2 Cluster**

Getting work done on the cluster

Screen%20Shot%202022-02-08%20at%209.39.32%20PM.png

On which resources will your job run?

https://www.hoffman2.idre.ucla.edu/Using-H2/Computing/Computing.html#computational-resources-on-the-hoffman2-cluster

Highp vs shared vs campus resources:

https://www.hoffman2.idre.ucla.edu/Using-H2/Computing/Computing.html#highp-vs-shared-vs-campus-jobs

Do I have access to highp resources?

To find out paste in a terminal connected to the clster the command (omitting the $ character indicative of the unix prompt):

$ myresources

what do you see?

What computational resources do I have access to?

Open a terminal on the Hoffman2 Cluster and issue:

$ myresources

if the first line of your ouput contains:

User joebruin is in the following resource group(s): campus

you do NOT have access to group-owned compute nodes and can only run for up to 24 hours on nodes owned by OARC/IDRE

if the first line of your output contains:

User joebruin is in the following resource group(s): gobruins evebruin

you have access to the nodes purchased by groups: gobruins and evebruin and you can run for up to 24 hours on shared queues and for up to 14 days when requesting to run on owned resources (highp mode)

Working interactively on the Hoffman2 Cluster

More on this on future presentations

Any work that will use substantial computational resources should be run on compute nodes and not on the login nodes.

To get an interactive session on one core of a compute node, from a terminal issue the following command(omitting the $ character indicative of the unix prompt):

$ qrsh

What happens?

(To terminate your interactive session, after the prompt returns, type: Control + d or logout)

What applications/software is already available on Hoffman2?

Refer to: https://www.hoffman2.idre.ucla.edu/Using-H2/Software/Software.html

Screenshot%202023-07-26%20at%209.34.40%20AM.png

Apps available via modules

To see what applications are available in the current hierarchy, at a terminal connected to Hoffman2 issue the command:

$ module av # press enter to scroll down and exit the view

To look for a specific software, for example R, issue the command:

$ modules_lookup -m R

Worked out example of python environment creation

 qrsh 
 module av python
 module av anaconda
 module av mamba
module load python/3.9.6
mkdir -p $HOME/.venv/3.9.6
python -m venv $HOME/.venv/3.9.6/NEWENV
source $HOME/.venv/3.9.6/NEWENV/bin/activate
# now install any needed python package, for example:
pip install numpy
# for example, update pip:
python -m pip install --upgrade pip